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Rubrics for Evaluating Open Education Resource (OER) Objects 

The following rubrics represent an evaluation system for objects found within Open Education 
Resources. An object could include images, applets, lessons, units, assessments and more. For 
the purpose of this evaluation, any component that can exist as a stand-alone qualifies as an 
object. The rubrics in this packet can be applied across content areas and object types. 

In general, the rubrics should be applied to the smallest meaningful unit. In some cases, this 
may be a single lesson or instructional support material, while in others it might be a complete 
unit of study or set of support materials. If multiple lessons are included in an OER, the 
reviewer needs to determine if all lessons will be examined, if only those lessons that deal with 
essential aspects of the curriculum are to be considered, or if it would be best to evaluate 
random lessons, looking at, for example, every third or fifth lesson. 

These rubrics are typically used to rate the potential, not actual, effectiveness of a particular 
object in a learning environment. Each rubric should be scored independently of the others 
using the following five scores that describe levels of potential quality, usefulness, or alignment 
to standards: 

3: Superior 

2: Strong 

1: Limited 

0: Very Weak / None 

N/A: Rubric Not Applicable 

The not applicable (N/A) rating should be used any time a particular rubric does not apply to 
the object being rated. This is not a pejorative score; it simply means it would be inappropriate 
to apply this rubric to this object. For example. Rubric IV: Quality of Assessment would not be 
applicable to an object that does not have an assessment component. 

The following rubrics are included: 

Rubric I. Degree of Alignment to Standards 

Rubric II. Quality of Explanation of the Subject Matter 

Rubric III. Utility of Materials Designed to Support Teaching 

Rubric IV. Quality of Assessment 

Rubric V. Quality of Technological Interactivity 

Rubric VI. Quality of Instructional and Practice Exercises 

Rubric VII. Qpportunities for Deeper Learning 

Rubric VIII. Assurance of Accessibility 
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Rubric I: Degree of Alignment to Standards 

This rubric is applied to learning objects that have suggested alignments to standards. It is used 
to rate the degree to which an individual object actually aligns to each proposed standard. The 
rubric was designed specifically for the Common Core State Standards, but can be used with 
any set of standards. Before the rubric can be applied, the assumption is that a user has 
proposed an alignment between the object and the selected standard(s). 

There are two major aspects of standards that are vital to a meaningful alignment review: 
content and performance expectations. It is important that the content addressed in the object 
matches the content addressed in each proposed standard. Evaluating the alignment of the 
performances required in both the object and the standard is equally essential and should be 
considered along with the content. 

Rubric I Scoring Guide: 

3: An object has superior alignment only if both of the following are true: 

• All of the content and performance expectations in the identified standard are 
completely addressed by the object. 

• The content and performance expectations of the identified standard are the focus of 
the object. While some objects may cover a range of standards that could potentially 
be aligned, for a superior alignment the content and performance expectations must 
not be a peripheral part of the object. 

2: An object has strong alignment for either one of two reasons: 

• Minor elements of the standard are not addressed in the object. 

• The content and performance expectations of the standard align to a minor part of the 
object. 

1: An object has limited alignment if a significant part of the content or performance 

expectations of the identified standard is not addressed in the object, as long as there is 
fidelity to the part it does cover. For example, an object that aligns to CCSS 2.NBT.2, 
"Count within 1000; skip-count by 5s, 10s, and 100s," but only addresses counting 
numbers to 500, would be considered to have limited alignment. The object aligns very 
closely with a limited part of the standard. 

0: An object has very weak alignment for either one of two reasons: 

• The object does not match the intended standards. 

• The object matches only to minimally important aspects of a standard. These objects 
will not typically be useful for instruction of core concepts and performances covered 
by the standard. 

N/A: This rubric does not apply for an object that has no suggested standards for alignment. 
For example, the rubric might not be applicable to a set of row data. 
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Rubric II: Quality of Explanation of the Subject Matter 

This rubric is applied to objects designed to explain subject matter. It is used to rate how 
thoroughly the subject matter is explained or otherwise revealed in the object. Teachers might 
use this object with a whole class, a small group, or an individual student. Students might use 
the object to self-tutor. For objects that are primarily intended for teacher use, the rubric is 
applied to the explanation of the subject matter not to the planning instructions for the 
teacher. 

Rubric II Scoring Guide: 

3: An object is rated superior for explanation of subject matter only if all of the following 

are true: 

• The object provides comprehensive information so effectively that the target audience 
should be able to understand the subject matter. 

• The object connects important associated concepts within the subject matter. For 
example, a lesson on multi-digit addition makes connections with place value, rather 
than simply showing how to add multi-digit numbers. Or a lesson designed to analyze 
how an author develops ideas across extended text would moke connections among 
the various developmental steps and the various purposes the author has for the text. 

• The object does not need to be augmented with additional explanation or materials. 

• The main ideas of the subject matter addressed in the object are clearly identified for 
the learner. 

2: An object is rated strong for explanation of subject matter if it explains the subject 

matter in a way that makes skills, procedures, concepts, and/or information 
understandable. It falls short of superior in that it does not make connections among 
important associated concepts within the subject matter. For example, a lesson on 
multi-digit addition may focus on the procedure and fail to connect it with place value. 

1: An object is rated limited for explanation of subject matter if it explains the subject 

matter correctly but in a limited way. This cursory treatment of the content is not 
sufficiently developed for a first-time learner of the content. The explanations are not 
thorough and would likely serve as a review for most learners. 

0: An object is rated very weak or no value for explanation of subject matter if its 

explanations are confusing or contain errors. There is little likelihood that this object 
will contribute to understanding. 

N/A: This rubric is not applicable (N/A) for an object that is not designed to explain subject 

matter, for example, a sheet of mathematical formulae or a map. It may be possible to 
apply the object in some way that aids a learner's understanding, but that is beyond any 
obvious or described purpose of the object. 
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Rubric III: Utility of Materials Designed to Support Teaching 

This rubric is applied to objects designed to support teachers in planning or presenting subject 
matter. The primary user would be a teacher. This rubric evaluates the potential utility of an 
object at the intended grade level for the majority of instructors. 

Rubric III Scoring Guide: 

3 : An object is rated superior for the utility of materials designed to support teaching only if 

all of the following are true: 

• The object provides materials that are comprehensive and easy to understand and use. 

• The object includes suggestions for ways to use the materials with a variety of learners. 
These suggestions include materials such as "common error analysis tips" and "precursor 
skills and knowledge" that go beyond the basic lesson or unit elements. 

• All objects and all components are provided and function as intended and described. For 
example, the time needed for lesson planning appears accurately estimated, materials 
lists are complete, and explanations make sense. 

• For larger objects like units, materials facilitate the use of a mix of instructional 
approaches (direct instruction, group work, investigations, etc.). 

2 : An object is rated strong for the utility of materials designed to support teaching if it 

offers materials that are comprehensive and easy to understand and use but falls short 
of "superior" for either one of two reasons: 

• The object does not include suggestions for ways to use the materials with a variety of 
learners (e.g., error analysis tips). 

• Some core components (e.g., directions) are underdeveloped in the object. 

1 : An object is rated limited for the utility of materials designed to support teaching if it 

includes a useful approach or idea to teach an important topic but falls short of "strong" 
for either one of two reasons: 

• The object is missing important elements (e.g. directions for some parts of a lesson are 
not included). 

• Important elements do not function as they are intended to (e.g. directions are 
unclear or practice exercises are missing or inadequate). Teachers would need to 
supplement this object to use it effectively. 

0 : An object is rated very weak or no value for the utility of materials designed to support 

teaching if it is confusing, contains errors, is missing important elements, or is for some 
other reason simply not useful, in spite of an intention to be used as a support for 
teachers in planning or preparation. 

N/A: This rubric is not applicable (N/A) for an object that is not designed to support teachers in 
planning and/or presenting subject matter. It may be possible that an educator could 
find an application for such an object during a lesson, but that would not be the intended 
use. 
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Rubric IV: Quality of Assessments 

This rubric is applied to those objects designed to determine what a student knows before, 
during, or after a topic is taught. When many assessment items are included in one object, as is 
often the case, the rubric is applied to the entire set. 

Rubric IV Scoring Guide: 

3: An object is rated superior for the quality of its assessments only if all of the following are 

true: 

• All of the skills and knowledge assessed align clearly to the content and performance 
expectations intended, as stated or implied in the object. 

• Nothing is assessed that is not included in the scope of intended material unless it is 
differentiated as extension material. 

• The most important aspects of the expectations are targeted and are given 
appropriate weight/attention in the assessment. 

• The assessment modes used in the object, such as selected response, long and short 
constructed response, or group work require the student to demonstrate proficiency 
in the intended concept/skill. 

• The level of difficulty is a result of the complexity of the subject-area content and 
performance expectations and of the degree of cognitive demand, rather than a result 
of unrelated issues (e.g. overly complex vocabulary used in math word problems). 

2: An object is rated strong for the quality of its assessments if it assesses all of the content 

and performance expectations intended, but the assessment modes used do not 
consistently offer the student opportunities to demonstrate proficiency in the intended 
concept/skill. 

1: An object is rated limited for the quality of its assessments if it assesses some of the 

content or performance expectations intended, as stated or implicit in the object, but 
omits some important content or performance expectations and/or fails to offer the 
student opportunities to demonstrate proficiency in the intended content/skills. 

0: An object is rated very weak or no value for the quality of its assessments if its 

assessments contain significant errors, do not assess important content/skills, are written 
in a way that is confusing to students, or are unsound for other reasons. 

N/A: This rubric is not applicable (N/A) for an object that is not designed to have an 

assessment component. Even if one might imagine ways an object could be used for 
assessment purposes, if it is not the intended purpose, not applicable is the appropriate 
score. 
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Rubric V: Quality of Technological Interactivity 

This rubric is applied to objects designed with a technology-based interactive component. It is 
used to rate the degree and quality of the interactivity of that component. "Interactivity" is 
used broadly to mean that the object responds to the user, in other words, it behaves 
differently based on what the user does. This is not a rating for technology in general, but for 
technological interactivity. The rubric does not apply to interaction between students, but 
rather to how the technology responds to the individual user. 

Rubric V Scoring Guide: 

3: An object, or interactive component of an object, is rated superior for the quality of its 

technological interactivity only if all of the following are true: 

• The object is responsive to student input in a way that creates an individualized 
learning experience. This means the object adapts to the user based on what s/he 
does, or the object allows the user some flexibility or individual control during the 
learning experience. 

• The interactive element is purposeful and directly related to learning. 

• The object is well-designed and easy to use, encouraging learner use. 

• The object appears to function flawlessly on the intended platform. 

2: An object, or interactive component of an object, is rated strong for the quality of its 

technological interactivity if it has an interactive feature that is purposeful and directly 
related to learning, but does not provide an individualized learning experience. Similarly 
to the superior objects, strong interactive objects must be well designed, easy-to-use, 
and function flawlessly on the intended platform. Some technological elements may not 
be directly related to the content but for a strong rating they must not detract from the 
learning experience. These kinds of interactive elements, including earning points or 
achieving levels for correct answers, might be designed to increase student motivation 
and to build content understanding by rewarding or entertaining the learner, and may 
extend the time the user engages with the content. 

1: An object, or interactive component of an object, is rated limited for the quality of its 

technological interactivity if its interactive element does not relate to the subject matter 
and may detract from the learning experience. These kinds of interactive elements may 
slightly increase motivation but do not provide strong support for understanding the 
subject matter addressed in the object. It is unlikely that this interactive feature will 
increase understanding or extend the time a user engages with the content. 

0: An object, or interactive component of an object, is rated very weak or no value for the 

quality of its technological interactivity if it has interactive features that are poorly 
conceived and/or executed. The interactive features might fail to operate as intended, 
distract the user, or unnecessarily take up user time. 
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N/A: This rubric is not applicable (N/A) for an object that does not have an interactive 

technological element. For example, the rubric does not apply if interaction with the 
object is limited to, for example, opening a user-selected PDF. 
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Rubric VI: Quality of Instructional and Practice Exercises 

This rubric is applied to objects that contain exercises designed to provide an opportunity to 
practice and strengthen specific skills and knowledge. The purpose of these exercises is to 
deepen understanding of subject matter and to routinize foundational skills and procedures. 
When concepts and skills are introduced, providing a sufficient number of exercises to support 
skill acquisition is critical. However when integrating skills in complex tasks, the number of 
exercise problems is less important than their richness. These types of practice opportunities 
may include as few as one or two instructional exercises designed to provide practice applying 
specific concepts and/or skills. Sets of practice exercises are treated as a single object, with the 
rubric applied to an entire group. 



Rubric VI Scoring Guide: 

3: An object is rated superior for the quality of its instructional and practice exercises only if 

all of the following are true: 

• The object offers more exercises than needed for the average student to facilitate 
mastery of the targeted skills, as stated or implied in the object. For complex tasks, 
one or two rich practice exercises may be considered more than enough. 

• The exercises are clearly written and supported by accurate answer keys or scoring 
guidelines as applicable. 

• There are a variety of exercise types and/or the exercises are available in a variety of 
formats, as appropriate to the targeted concepts and skills. For more complex practice 
exercises the formats used provide an opportunity for the learner to integrate a 
variety of skills. 

2: An object is rated strong for the quality of its instructional and practice exercises if it 

offers only a sufficient number of well-written exercises to facilitate mastery of targeted 
skills, which are supported by accurate answer keys or scoring guidelines, but there is 
little variety of exercise types or formats. 

1: An object is rated limited for the quality of its instructional and practice exercises if it has 

some, but too few exercises to facilitate mastery of the targeted skills, is without answer 
keys, and provides no variation in type or format. 

0: An object is rated very weak or no value for the quality of its instructional and practice 

exercises if the exercises provided do not facilitate mastery of the targeted skills, contain 
errors, or are unsound for other reasons. 

N/A: This rubric is not applicable (N/A) to an object that does not include opportunities to 
practice targeted skills. 
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Rubric VII: Opportunities for Deeper Learning 

This rubric is applied to objects designed to engage learners in at least one of the following 
deeper learning skills, which can be applied across all content areas: 

• Think critically and solve complex problems. 

• Work collaboratively. 

• Communicate effectively. 

• Learn how to learn. 

• Reason abstractly. 

• Construct viable arguments and critique the reasoning of others. 

• Apply discrete knowledge and skills to real-world situations. 

• Construct, use, or analyze models. 

Rubric VII Scoring Guide: 

3: An object is rated superior for its opportunities for deeper learning only if all of the 

following are true: 

• At least three of the deeper learning skills from the list identified in this rubric are 
required in the object. 

• The object offers a range of cognitive demand that is appropriate and supportive of 
the material. 

• Appropriate scaffolding and direction are provided. 

2: An object is rated strong for its opportunities for deeper learning if it includes one or two 

deeper learning skills identified in this rubric. For example, the object might involve a 
complex problem that reguires abstract reasoning skills to reach a solution. 

1: An object is rated limited for its opportunities for deeper learning if it includes one 

deeper learning skill identified in the rubric but is missing clear guidance on how to tap 
into the various aspects of deeper learning. For example, an object might include a 
provision for learners to collaborate, but the process and product are unclear. 

0: An object is rated very weak for its opportunities for deeper learning if it appears to be 

designed to provide some of the deeper learning opportunities identified in this rubric, 
but it is not useful as it is presented. For example, the object might be based on poorly 
formulated problems and/or unclear directions, making it unlikely that this lesson or 
activity will lead to skills like critical thinking, abstract reasoning, constructing arguments, 
or modeling. 

N/A: This rubric is not applicable (N/A) to an object that does not appear to be designed to 
provide the opportunity for deeper learning, even though one might imagine how it 
could be used to do so. 
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Rubric VIII: Assurance of Accessibility Standards 

This rubric is used to assure materials are accessible to all students, including students 
identified as blind, visually impaired or print disabled, and those students who may qualify 
under the Chafee Amendment to the U.S. 1931 Act to Provide Books to the Adult Blind as 
Amended. It was developed to assess compliance with U.S. standards and requirements, but 
could be adapted to accommodate differences in other sets of requirements internationally. 

Accessibility is critically important for all learners and should be considered in the design of all 
online materials. Identification of certain characteristics will assist in determining if materials 
will be fully accessible for all students. Assurance that materials are compliant with the 
standards, recommendations, and guidelines specified assists educators in the selection and 
use of accessible versions of materials that can be used with all students, including those with 
different kinds of challenges and assistive devices. 

The Assurance of Accessibility Standards Rubric does not ask reviewers to make a judgment on 
the degree of object quality. Instead, it requests that a determination (yes/no) of characteristics 
be made that, together with assurance of specific Standards, may determine the degree to 
which the materials are accessible. Only those who feel qualified to make judgments about an 
object's accessibility should use this rubric. 

Rubric VIII Scoring Guide (see table next page): 

Yes: The object displays the characteristic or complies with the standards, 

recommendations or guidelines. 

No: The object does NOT display the characteristic or comply with the standards, 

recommendations or guidelines. 

Comment: Comments on Rubric 8 Object determination may include notes that describe 

the reason materials do not comply with the standard, recommendations or 
guidelines or further description that may clarify the characteristics of the 
object. 
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YES/NO/NA 


Comment or 
Explanation 


Organization that 
Maintains the Standard 


Available in Tagged PDF Format 






Adobe 


Available in ePUB Format 






International Digital 
Publishing Form 


Accessible Course within an Open Learning 
Management System (LMS) 






Moodle 


Accessible Course within another Learning 
Management System (LMS) 






LMS Provider 


Available in an accessible media format 
and includes alternate text or subtitles 






Provider or Publisher 


Includes alternative text (image) 






Provider or Publisher 


Includes captions and subtitles (video) 






Provider or Publisher 


Includes flash accessibility functions (SWF) 






Adobe 


Includes functionality that provide 
accessibility 






Provider or Publisher 


Complies with WC3 WCAG2 
Recommendations for web pages 






WC3 Recommendations 


Compliant with Section 508 of the 
Rehabilitation Act 






US Government 


Is accessible as determined by Utah State 
WebAIM Web Accessibility Evaluation 
(WAVE) Tool 






Utah State WebAIM 


Available in National Accessible 
Instructional Materials Standard (NIMAS) 
Format - Accessible XM L 






NIMAS Center at CAST 


Complies with Audio/Video Cassette 
Production Standards 






ITA Standards 


Complies with DVD/DVD-ROM Production 
Standards 






DVD Forum 
Specifications 


Complies with Blue-ray Disk Production 
Standards 






UDF 2.5 - Blue-ray Disk 
Association 


Complies with NCAM Guidelines for 
Movies, Web and Multimedia 






NCAM Guidelines 



Additional references for accessibility: 

Accessible Instructional Materials at the Center for Applied Special Technology 

- http://aim.cast.org/learn/e-resources/accessibility resources 
National Center for Accessible Media 

- http://ncam.wgbh.org/about/accessibility-links 
Accessible Publishing: Best Practice Guidelines for Publishers. 

-PDF: http://www.editeur.org/109/Enabling-Technologies-Framework/ 

-HTML: http://www.editeur.org/files/Collaborations/Accessibility/WIPQ.html 
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